Learning Objectives

After completing this lesson, you’ll be able to:

• Define FME Flow parameters and their purpose. 
• Link to FME Flow parameters in a workspace.
• Run a workspace that uses FME Flow parameters. 

In this lesson, you will:

• Optional: Watch a demonstration video (if you have not attended the live training).
• Scroll down to read the activity instructions below and follow the steps in your lab.
• Complete the quiz at the end of this lesson.
• Click 'Next' to mark the lesson complete.

Learning Objectives

After completing this lesson, you’ll be able to:

• Define FME Flow parameters and their purpose. 
• Link to FME Flow parameters in a workspace.
• Run a workspace that uses FME Flow parameters. 

In this lesson, you will:

• Optional: Watch a demonstration video (if you have not attended the live training).
• Scroll down to read the activity instructions below and follow the steps in your lab.
• Complete the quiz at the end of this lesson.
• Click 'Next' to mark the lesson complete.

Resources

• Starting workspace | C:\FMEData\Workspaces\DeployWorkflowsWithFMEFlow\use-fme-flow-parameters.fmw
• Complete workspace | C:\FMEData\Workspaces\DeployWorkflowsWithFMEFlow\use-fme-flow-parameters--complete.fmw

FME Flow Parameters

FME Flow parameters reference FME Flow system settings to customize and control workspaces run from FME Flow. FME Workbench lists the FME Flow parameters in the Navigator window for easy insertion into workflows. 

Some commonly used FME Flow parameters include:

• FME_SHAREDRESOURCE_DATA references the FME Flow Resources folder, which stores data files that you may share between workspaces and users. This parameter helps read and write data to Resources. 
• FME_SECURITY_USER references the FME Flow user who runs the workspace, and FME_SECURITY_ROLE is the role the user belongs to. You may use the parameter to help monitor FME Flow usage by user and role, filter job results, and log workspace-user information. 
• FME_JOB_ID references the unique ID for each job run on FME Flow. You may use this parameter to name output files, log jobs, track jobs, and troubleshoot. 
• FME_SERVER_WEB_URL specifies the Web Application Server root URL that invokes the workspace to run. 

You can find a description of all FME Flow parameters in our FME Flow Parameters documentation. 

You can learn more about managing data in the FME Flow Resources with the FME_SHAREDRESOURCE_DATA parameter in Manage FME Flow Data and Connections. 

Linking FME Flow Parameters

FME Flow parameters are linked to readers, writers, and transformers, just as existing user parameters are. Here's an example of using the FME_SECURITY_ROLES parameter in a Tester to filter for users who belong to the fmeadmin or fmesuperuser roles on FME Flow. You might expand this workflow to write specific data to the user if they belong to one of those roles, and to exclude it otherwise. 

Like user parameters in FME Workbench, FME Flow parameters can be hardcoded into workspace parameters following the $(FME_FLOW_PARAMETER) format. Here's an example where the output CSV location references the FME Flow Resources Data folder using the $(FME_SHAREDRESOURCE_DATA) parameter. 

By default, FME Workbench populates some FME Flow parameters once you establish a successful connection to FME Flow. These include FME_SERVER_WEB_URL, FME_SERVER_RUNTIME_USER, and FME_SERVER_HOST. Otherwise, when you run workspaces in FME Workbench that reference FME Flow parameters, you need to set the parameter value manually. However, when the workspace runs on FME Flow using these parameters, FME Flow automatically fills in the value based on the FME Flow system settings. Here's an example where the FME_SERVER_WEB_URL parameter is referenced in an HTTP Caller to make a GET request to the FME Flow REST API. If you move the workspace to different FME Flow instances, the parameter will update to use the correct URL value for each FME Flow. 

While running workspaces on FME Workbench can be useful for testing and debugging, you do not need to set these values to run the workspace only with FME Flow, as it will automatically set and use the correct corresponding value. 

Exercise

Jennifer continues updating the workspaces she will deploy to FME Flow. She's working on one that clips election voting locations by neighborhood and writes a list of voting places for each neighborhood. This workspace currently writes the data to a local file system, and Jennifer shares it with users who need it for their FME workflows. However, others can easily access it if it were written to the FME Flow Resources, so other users' workflows can read the data from Resources. To accomplish this, Jennifer will use the $(FME_SHAREDRESOURCE_DATA) parameter to write the output data to the FME Flow Resources Data folder. 

For this exercise, follow the steps below to help Jennifer configure the workspace to use the $(FME_SHAREDRESOURCE_DATA) parameter. 

1) Open Starting Workspace

Open the starting workspace (C:\FMEData\Workspaces\DeployWorkflowsWithFMEFlow\use-fme-flow-parameters.fmw) in FME Workbench. The workspace reads the Voting Places and Neighborhoods, clips the data by neighborhood, and writes the clipped data to a Microsoft Excel file with a different sheet for each neighborhood. 

2) Edit Writer Parameters

Expand the VotingPlaces [XLSXW] writer parameters in the Navigator and double-click the Destination Microsoft Excel File parameter. 

Use the drop-down arrow to select the $(FME_SHAREDRESOURCE_DATA) FME Flow parameter and then add \Output\VotingPlaces.xlsx.

Now that the workspace writes data using the FME Flow parameter, click OK to close the writer settings.

Since you have not set the value for the $(FME_SHAREDRESOURCE_DATA) in FME Workbench, you won't run the workspace before publishing it to FME Flow. 

3) Publish to FME Flow

Publish the workspace to the Training repository and register it with the Job Submitter service.

Publish to FME Flow

Once you build your workflows in FME Workbench, you need to transfer them to FME Flow through the publishing process. To publish a workspace to FME Flow, you need to complete all the steps in the publishing wizard.  

1. Click Publish in the FME Flow Connection toolbar. The publishing wizard window will open.

2. Confirm your FME Flow Connection lists the correct connection and successfully connects to your FME Flow.

3. Select a repository to house the workspace on FME Flow from the drop-down options. To create a new repository, click New and enter its details. 

4. Optionally, you may Upload data files to publish the workspace's source data files to FME Flow along with the workspace. You can control which files to upload from the Select Files... option. 

5. Click Next to move on to the next step.

6. If your workspace uses any connections, the Upload Connections section will appear. You can select whether to publish the connections to FME Flow. 

7. Register Services is the last publishing step. Select the transformation services you want your workspace to be able to use on FME Flow. 

8. Click Publish. 

9. The translation log generates a summary of publishing and provides a direct link to open the workspace on FME Flow. 

For more information on the FME Flow services and publishing to FME Flow, see Deploy Workspaces on FME Flow. 

4) Run workspace

Navigate to the FME Flow Run Workspace page and select your recently published workspace. The Destination Microsoft Excel File parameter shows the $(FME_SHAREDRESOURCE_DATA) parameter you referenced in the writer settings. 

Don't change any settings, then click Run. The workspace runs and processes the Voting Places data. 

5) Inspect Resources for Output

Open Resources, and select the Data and then Output folders.

Here, you'll see the workspace wrote the VotingPlaces.xlsx file a moment ago. The workspace used the $(FME_SHAREDRESOURCE_DATA) parameter to reference the FME Flow Resources location for the output. Jennifer's next steps will be to automate this workflow and run it on a schedule to keep the data in Resources up to date for other users to access. 

Resources

• Starting workspace | C:\FMEData\Workspaces\DeployWorkflowsWithFMEFlow\use-fme-flow-parameters.fmw
• Complete workspace | C:\FMEData\Workspaces\DeployWorkflowsWithFMEFlow\use-fme-flow-parameters--complete.fmw

FME Flow Parameters

FME Flow parameters reference FME Flow system settings to customize and control workspaces run from FME Flow. FME Workbench lists the FME Flow parameters in the Navigator window for easy insertion into workflows. 

Some commonly used FME Flow parameters include:

• FME_SHAREDRESOURCE_DATA references the FME Flow Resources folder, which stores data files that you may share between workspaces and users. This parameter helps read and write data to Resources. 
• FME_SECURITY_USER references the FME Flow user who runs the workspace, and FME_SECURITY_ROLE is the role the user belongs to. You may use the parameter to help monitor FME Flow usage by user and role, filter job results, and log workspace-user information. 
• FME_JOB_ID references the unique ID for each job run on FME Flow. You may use this parameter to name output files, log jobs, track jobs, and troubleshoot. 
• FME_SERVER_WEB_URL specifies the Web Application Server root URL that invokes the workspace to run. 

You can find a description of all FME Flow parameters in our FME Flow Parameters documentation. 

You can learn more about managing data in the FME Flow Resources with the FME_SHAREDRESOURCE_DATA parameter in Manage FME Flow Data and Connections. 

Linking FME Flow Parameters

FME Flow parameters are linked to readers, writers, and transformers, just as existing user parameters are. Here's an example of using the FME_SECURITY_ROLES parameter in a Tester to filter for users who belong to the fmeadmin or fmesuperuser roles on FME Flow. You might expand this workflow to write specific data to the user if they belong to one of those roles, and to exclude it otherwise. 

Like user parameters in FME Workbench, FME Flow parameters can be hardcoded into workspace parameters following the $(FME_FLOW_PARAMETER) format. Here's an example where the output CSV location references the FME Flow Resources Data folder using the $(FME_SHAREDRESOURCE_DATA) parameter. 

By default, FME Workbench populates some FME Flow parameters once you establish a successful connection to FME Flow. These include FME_SERVER_WEB_URL, FME_SERVER_RUNTIME_USER, and FME_SERVER_HOST. Otherwise, when you run workspaces in FME Workbench that reference FME Flow parameters, you need to set the parameter value manually. However, when the workspace runs on FME Flow using these parameters, FME Flow automatically fills in the value based on the FME Flow system settings. Here's an example where the FME_SERVER_WEB_URL parameter is referenced in an HTTP Caller to make a GET request to the FME Flow REST API. If you move the workspace to different FME Flow instances, the parameter will update to use the correct URL value for each FME Flow. 

While running workspaces on FME Workbench can be useful for testing and debugging, you do not need to set these values to run the workspace only with FME Flow, as it will automatically set and use the correct corresponding value. 

Exercise

Jennifer continues updating the workspaces she will deploy to FME Flow. She's working on one that clips election voting locations by neighborhood and writes a list of voting places for each neighborhood. This workspace currently writes the data to a local file system, and Jennifer shares it with users who need it for their FME workflows. However, others can easily access it if it were written to the FME Flow Resources, so other users' workflows can read the data from Resources. To accomplish this, Jennifer will use the $(FME_SHAREDRESOURCE_DATA) parameter to write the output data to the FME Flow Resources Data folder. 

For this exercise, follow the steps below to help Jennifer configure the workspace to use the $(FME_SHAREDRESOURCE_DATA) parameter. 

1) Open Starting Workspace

Open the starting workspace (C:\FMEData\Workspaces\DeployWorkflowsWithFMEFlow\use-fme-flow-parameters.fmw) in FME Workbench. The workspace reads the Voting Places and Neighborhoods, clips the data by neighborhood, and writes the clipped data to a Microsoft Excel file with a different sheet for each neighborhood. 

2) Edit Writer Parameters

Expand the VotingPlaces [XLSXW] writer parameters in the Navigator and double-click the Destination Microsoft Excel File parameter. 

Use the drop-down arrow to select the $(FME_SHAREDRESOURCE_DATA) FME Flow parameter and then add \Output\VotingPlaces.xlsx.

Now that the workspace writes data using the FME Flow parameter, click OK to close the writer settings.

Since you have not set the value for the $(FME_SHAREDRESOURCE_DATA) in FME Workbench, you won't run the workspace before publishing it to FME Flow. 

3) Publish to FME Flow

Publish the workspace to the Training repository and register it with the Job Submitter service.

Publish to FME Flow

Once you build your workflows in FME Workbench, you need to transfer them to FME Flow through the publishing process. To publish a workspace to FME Flow, you need to complete all the steps in the publishing wizard.  

1. Click Publish in the FME Flow Connection toolbar. The publishing wizard window will open.

2. Confirm your FME Flow Connection lists the correct connection and successfully connects to your FME Flow.

3. Select a repository to house the workspace on FME Flow from the drop-down options. To create a new repository, click New and enter its details. 

4. Optionally, you may Upload data files to publish the workspace's source data files to FME Flow along with the workspace. You can control which files to upload from the Select Files... option. 

5. Click Next to move on to the next step.

6. If your workspace uses any connections, the Upload Connections section will appear. You can select whether to publish the connections to FME Flow. 

7. Register Services is the last publishing step. Select the transformation services you want your workspace to be able to use on FME Flow. 

8. Click Publish. 

9. The translation log generates a summary of publishing and provides a direct link to open the workspace on FME Flow. 

For more information on the FME Flow services and publishing to FME Flow, see Deploy Workspaces on FME Flow. 

4) Run workspace

Navigate to the FME Flow Run Workspace page and select your recently published workspace. The Destination Microsoft Excel File parameter shows the $(FME_SHAREDRESOURCE_DATA) parameter you referenced in the writer settings. 

Don't change any settings, then click Run. The workspace runs and processes the Voting Places data. 

5) Inspect Resources for Output

Open Resources, and select the Data and then Output folders.

Here, you'll see the workspace wrote the VotingPlaces.xlsx file a moment ago. The workspace used the $(FME_SHAREDRESOURCE_DATA) parameter to reference the FME Flow Resources location for the output. Jennifer's next steps will be to automate this workflow and run it on a schedule to keep the data in Resources up to date for other users to access.